docs(mcp): document long-running tool calls and Tasks roadmap - #30521
docs(mcp): document long-running tool calls and Tasks roadmap#30521mateo-berri wants to merge 1 commit into
Conversation
Capture how the MCP gateway streams progress for long-running tool calls today (progressToken forwarding over the stateful StreamableHTTP transport restored in #26857) and why the Tasks lifecycle is not implemented yet, with a pointer to the LIT-3784 migration plan.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
🚅 Hi, thanks for the PR! I'm Agent Shin, the automated triage bot for this repository. What's this and why am I getting it? I read the description against our contribution rubric. Here's how it lined up: What you got right:
What's still missing:
If the description isn't updated in the next 24 hours, I'll auto-close this PR. That's not us saying we don't care about the change; we want the open-PR list to mirror what a maintainer can act on right now, so contributors don't get lost in a backlog. A closed PR is a soft "park this for later," not a rejection. Take your time; everything below still works after the close. During the grace period: just update the PR description with the missing pieces. No need to ping me; I'll re-check on the next sweep and skip the auto-close if it now passes. See what counts as QA proof for the full rubric (a linked issue alone isn't enough; it covers context, not proof). If the PR does get auto-closed in 24 hours, you still have easy recovery paths:
Internal BerriAI contributors: this rubric doesn't apply to you; ping a maintainer. (I'm an LLM, so I'm not infallible. If you think I got this wrong, ping a maintainer; they'll override me.) |
1 participant
Relevant issues
Came out of a LANL support thread asking whether the MCP gateway supports the new Tasks extension and, if not, what the recommended path is for long-running tool calls
Linear ticket
LIT-3784 (Align MCP server with the 2026-07-28 stateless spec: Tasks extension + session removal)
Pre-Submission checklist
make test-unitScreenshots / Proof of Fix
Docs-only change, so there is no runtime behavior to exercise against a live proxy. The artifact is the rendered README at
litellm/proxy/_experimental/mcp_server/README.mdType
📖 Documentation
Changes
Adds a developer reference next to the MCP server that captures how the gateway handles long-running tool calls today and where the work is headed. The knowledge currently only lives in the
server.pyprogress-forwarding path and in the support thread, so this writes it downWhat it documents:
progressTokenin thetools/call_meta; the gateway captures it and relays upstream progress back to the host session viahost_progress_callback/forward_progress. This needs the stateful StreamableHTTP transport that feat(mcp): support stateless and stateful clients via session-id routing #26857 restored, and it is bounded by the connection/TTL windowtasks/get/tasks/update/tasks/cancel), reshaped into an extension in the2026-07-28spec RC and gated on themcpPython SDK v2 line; tracked in LIT-3784The site docs tree is not present in this branch, so this lands as a code-adjacent README rather than a Docusaurus page; the customer-facing docs page should be added wherever that site builds from
https://claude.ai/code/session_01KDdQ8YqNVM3Hh4a92QBhfq
Generated by Claude Code